{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "complicated-voluntary",
   "metadata": {},
   "source": [
    "# Choosing window features using LASSO"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "promotional-worcester",
   "metadata": {},
   "source": [
    "[Feature Engineering for Time Series Forecasting](https://www.trainindata.com/p/feature-engineering-for-forecasting)\n",
    "\n",
    "In this notebook, we will create a large number of window features using a pipeline and then use LASSO as a feature selection method to reduce the number of features we use.\n",
    "\n",
    "\n",
    "## Data set synopsis\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "876b948e-2e7f-4321-9ff8-1b9734f4be6f",
   "metadata": {},
   "source": [
    "We will work with the hourly electricity demand dataset. It is the electricity demand for the state of Victora in Australia from 2002 to the start of 2015. \n",
    "\n",
    "For instructions on how to download, prepare, and store the dataset, refer to notebook number 4, in the folder \"01-Create-Datasets\" from this repo.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "hybrid-fever",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_context(\"talk\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "confirmed-things",
   "metadata": {},
   "source": [
    "# Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "intended-logan",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\n",
    "    \"../Datasets/victoria_electricity_demand.csv\",\n",
    "    usecols=[\"demand\", \"temperature\", \"date_time\"],\n",
    "    index_col=[\"date_time\"],\n",
    "    parse_dates=[\"date_time\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d5252f36-7633-46f0-8f9b-d4707fac9330",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For this demo we will use a subset of the data\n",
    "data = data.loc[\"2010\":]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "functional-steal",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>demand</th>\n",
       "      <th>temperature</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2010-01-01 00:00:00</th>\n",
       "      <td>8314.448682</td>\n",
       "      <td>21.525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-01-01 01:00:00</th>\n",
       "      <td>8267.187296</td>\n",
       "      <td>22.400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-01-01 02:00:00</th>\n",
       "      <td>7394.528444</td>\n",
       "      <td>22.150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-01-01 03:00:00</th>\n",
       "      <td>6952.047520</td>\n",
       "      <td>21.800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-01-01 04:00:00</th>\n",
       "      <td>6867.199634</td>\n",
       "      <td>20.250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          demand  temperature\n",
       "date_time                                    \n",
       "2010-01-01 00:00:00  8314.448682       21.525\n",
       "2010-01-01 01:00:00  8267.187296       22.400\n",
       "2010-01-01 02:00:00  7394.528444       22.150\n",
       "2010-01-01 03:00:00  6952.047520       21.800\n",
       "2010-01-01 04:00:00  6867.199634       20.250"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46cb4696-4300-45e6-9b91-aa99e8f7b938",
   "metadata": {},
   "source": [
    "# Create lag and window features using a pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "94aba288-e0a3-4e9f-a451-9184b1ba73bf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from feature_engine.timeseries.forecasting import LagFeatures, WindowFeatures, ExpandingWindowFeatures\n",
    "from feature_engine.imputation import DropMissingData\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f6b6346f-55b6-4bee-ba9c-6a67eb6173b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = data.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "740d4dc6-c527-4558-bf98-e3253839fb2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lag features\n",
    "lag_transformer = LagFeatures(variables=[\"demand\", \"temperature\"],\n",
    "                              periods=[1, 2, 3, 24, 24 * 7])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f38ea578-f557-4189-b47c-3bcc44d6ac0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Window features\n",
    "window_transformer = WindowFeatures(\n",
    "    variables=[\"demand\", \"temperature\"],\n",
    "    functions=[\"mean\", \"std\", \"kurt\", \"skew\"],\n",
    "    window=[24, 24 * 7, 24 * 7 * 4, 24 * 7 * 4 * 12],\n",
    "    periods=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f4d8e5ce-1877-4299-b7f7-73a97714d9d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expanding features\n",
    "expanding_window_transformer = ExpandingWindowFeatures(\n",
    "    variables=[\"demand\"], \n",
    "    functions=[\"mean\", \"std\", \"kurt\", \"skew\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b371787b-fb4c-4351-8b82-bc84a917a0d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop missing data introduced by window and lag features\n",
    "imputer = DropMissingData()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "262e031c-9840-46ef-9d40-da1a16ffc423",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>demand</th>\n",
       "      <th>temperature</th>\n",
       "      <th>demand_lag_1</th>\n",
       "      <th>temperature_lag_1</th>\n",
       "      <th>demand_lag_2</th>\n",
       "      <th>temperature_lag_2</th>\n",
       "      <th>demand_lag_3</th>\n",
       "      <th>temperature_lag_3</th>\n",
       "      <th>demand_lag_24</th>\n",
       "      <th>temperature_lag_24</th>\n",
       "      <th>...</th>\n",
       "      <th>demand_window_8064_kurt</th>\n",
       "      <th>demand_window_8064_skew</th>\n",
       "      <th>temperature_window_8064_mean</th>\n",
       "      <th>temperature_window_8064_std</th>\n",
       "      <th>temperature_window_8064_kurt</th>\n",
       "      <th>temperature_window_8064_skew</th>\n",
       "      <th>demand_expanding_mean</th>\n",
       "      <th>demand_expanding_std</th>\n",
       "      <th>demand_expanding_kurt</th>\n",
       "      <th>demand_expanding_skew</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2010-12-03 00:00:00</th>\n",
       "      <td>7650.165828</td>\n",
       "      <td>17.825</td>\n",
       "      <td>8311.641438</td>\n",
       "      <td>18.325</td>\n",
       "      <td>8194.758870</td>\n",
       "      <td>18.650</td>\n",
       "      <td>8810.225934</td>\n",
       "      <td>19.000</td>\n",
       "      <td>7594.965872</td>\n",
       "      <td>18.500</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.103898</td>\n",
       "      <td>0.422491</td>\n",
       "      <td>15.744401</td>\n",
       "      <td>5.742699</td>\n",
       "      <td>0.659439</td>\n",
       "      <td>0.778176</td>\n",
       "      <td>9842.090580</td>\n",
       "      <td>1804.188369</td>\n",
       "      <td>-0.103898</td>\n",
       "      <td>0.422491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-12-03 01:00:00</th>\n",
       "      <td>7927.140368</td>\n",
       "      <td>17.850</td>\n",
       "      <td>7650.165828</td>\n",
       "      <td>17.825</td>\n",
       "      <td>8311.641438</td>\n",
       "      <td>18.325</td>\n",
       "      <td>8194.758870</td>\n",
       "      <td>18.650</td>\n",
       "      <td>7914.538048</td>\n",
       "      <td>18.375</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.104160</td>\n",
       "      <td>0.422422</td>\n",
       "      <td>15.743942</td>\n",
       "      <td>5.742385</td>\n",
       "      <td>0.660363</td>\n",
       "      <td>0.778423</td>\n",
       "      <td>9841.818798</td>\n",
       "      <td>1804.241597</td>\n",
       "      <td>-0.104074</td>\n",
       "      <td>0.422631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-12-03 02:00:00</th>\n",
       "      <td>7327.146056</td>\n",
       "      <td>17.675</td>\n",
       "      <td>7927.140368</td>\n",
       "      <td>17.850</td>\n",
       "      <td>7650.165828</td>\n",
       "      <td>17.825</td>\n",
       "      <td>8311.641438</td>\n",
       "      <td>18.325</td>\n",
       "      <td>7321.428112</td>\n",
       "      <td>17.875</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.104297</td>\n",
       "      <td>0.422397</td>\n",
       "      <td>15.743378</td>\n",
       "      <td>5.741955</td>\n",
       "      <td>0.661546</td>\n",
       "      <td>0.778705</td>\n",
       "      <td>9841.581422</td>\n",
       "      <td>1804.255693</td>\n",
       "      <td>-0.104144</td>\n",
       "      <td>0.422815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-12-03 03:00:00</th>\n",
       "      <td>7088.725786</td>\n",
       "      <td>17.625</td>\n",
       "      <td>7327.146056</td>\n",
       "      <td>17.675</td>\n",
       "      <td>7927.140368</td>\n",
       "      <td>17.850</td>\n",
       "      <td>7650.165828</td>\n",
       "      <td>17.825</td>\n",
       "      <td>7045.315052</td>\n",
       "      <td>17.425</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.104314</td>\n",
       "      <td>0.422377</td>\n",
       "      <td>15.742823</td>\n",
       "      <td>5.741552</td>\n",
       "      <td>0.662685</td>\n",
       "      <td>0.778992</td>\n",
       "      <td>9841.269728</td>\n",
       "      <td>1804.361038</td>\n",
       "      <td>-0.104420</td>\n",
       "      <td>0.422872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010-12-03 04:00:00</th>\n",
       "      <td>7458.314830</td>\n",
       "      <td>17.625</td>\n",
       "      <td>7088.725786</td>\n",
       "      <td>17.625</td>\n",
       "      <td>7327.146056</td>\n",
       "      <td>17.675</td>\n",
       "      <td>7927.140368</td>\n",
       "      <td>17.850</td>\n",
       "      <td>7396.896962</td>\n",
       "      <td>17.275</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.104304</td>\n",
       "      <td>0.422436</td>\n",
       "      <td>15.742305</td>\n",
       "      <td>5.741194</td>\n",
       "      <td>0.663728</td>\n",
       "      <td>0.779267</td>\n",
       "      <td>9840.928560</td>\n",
       "      <td>1804.509421</td>\n",
       "      <td>-0.104740</td>\n",
       "      <td>0.422842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 19:00:00</th>\n",
       "      <td>9596.777060</td>\n",
       "      <td>28.350</td>\n",
       "      <td>9979.909902</td>\n",
       "      <td>30.850</td>\n",
       "      <td>10258.585392</td>\n",
       "      <td>31.550</td>\n",
       "      <td>10019.921572</td>\n",
       "      <td>31.250</td>\n",
       "      <td>9980.108798</td>\n",
       "      <td>19.700</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.398990</td>\n",
       "      <td>0.291512</td>\n",
       "      <td>15.987450</td>\n",
       "      <td>5.252119</td>\n",
       "      <td>1.053306</td>\n",
       "      <td>0.793359</td>\n",
       "      <td>9463.653128</td>\n",
       "      <td>1752.041445</td>\n",
       "      <td>0.252346</td>\n",
       "      <td>0.499886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 20:00:00</th>\n",
       "      <td>8883.230296</td>\n",
       "      <td>22.200</td>\n",
       "      <td>9596.777060</td>\n",
       "      <td>28.350</td>\n",
       "      <td>9979.909902</td>\n",
       "      <td>30.850</td>\n",
       "      <td>10258.585392</td>\n",
       "      <td>31.550</td>\n",
       "      <td>9411.874558</td>\n",
       "      <td>18.750</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.398995</td>\n",
       "      <td>0.291267</td>\n",
       "      <td>15.988492</td>\n",
       "      <td>5.253737</td>\n",
       "      <td>1.051447</td>\n",
       "      <td>0.793594</td>\n",
       "      <td>9463.656071</td>\n",
       "      <td>1752.022191</td>\n",
       "      <td>0.252414</td>\n",
       "      <td>0.499886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 21:00:00</th>\n",
       "      <td>8320.260550</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8883.230296</td>\n",
       "      <td>22.200</td>\n",
       "      <td>9596.777060</td>\n",
       "      <td>28.350</td>\n",
       "      <td>9979.909902</td>\n",
       "      <td>30.850</td>\n",
       "      <td>8653.510960</td>\n",
       "      <td>18.300</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.398790</td>\n",
       "      <td>0.291143</td>\n",
       "      <td>15.988814</td>\n",
       "      <td>5.254039</td>\n",
       "      <td>1.050537</td>\n",
       "      <td>0.793438</td>\n",
       "      <td>9463.643240</td>\n",
       "      <td>1752.004951</td>\n",
       "      <td>0.252485</td>\n",
       "      <td>0.499911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 22:00:00</th>\n",
       "      <td>8110.055916</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8320.260550</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8883.230296</td>\n",
       "      <td>22.200</td>\n",
       "      <td>9596.777060</td>\n",
       "      <td>28.350</td>\n",
       "      <td>8256.683092</td>\n",
       "      <td>18.150</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.398505</td>\n",
       "      <td>0.291119</td>\n",
       "      <td>15.988796</td>\n",
       "      <td>5.254028</td>\n",
       "      <td>1.050578</td>\n",
       "      <td>0.793450</td>\n",
       "      <td>9463.617965</td>\n",
       "      <td>1751.993833</td>\n",
       "      <td>0.252528</td>\n",
       "      <td>0.499947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 23:00:00</th>\n",
       "      <td>8519.368752</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8110.055916</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8320.260550</td>\n",
       "      <td>18.900</td>\n",
       "      <td>8883.230296</td>\n",
       "      <td>22.200</td>\n",
       "      <td>8716.498334</td>\n",
       "      <td>17.800</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.398214</td>\n",
       "      <td>0.291138</td>\n",
       "      <td>15.988808</td>\n",
       "      <td>5.254035</td>\n",
       "      <td>1.050551</td>\n",
       "      <td>0.793442</td>\n",
       "      <td>9463.588045</td>\n",
       "      <td>1751.986027</td>\n",
       "      <td>0.252556</td>\n",
       "      <td>0.499983</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>37176 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                          demand  temperature  demand_lag_1  \\\n",
       "date_time                                                     \n",
       "2010-12-03 00:00:00  7650.165828       17.825   8311.641438   \n",
       "2010-12-03 01:00:00  7927.140368       17.850   7650.165828   \n",
       "2010-12-03 02:00:00  7327.146056       17.675   7927.140368   \n",
       "2010-12-03 03:00:00  7088.725786       17.625   7327.146056   \n",
       "2010-12-03 04:00:00  7458.314830       17.625   7088.725786   \n",
       "...                          ...          ...           ...   \n",
       "2015-02-28 19:00:00  9596.777060       28.350   9979.909902   \n",
       "2015-02-28 20:00:00  8883.230296       22.200   9596.777060   \n",
       "2015-02-28 21:00:00  8320.260550       18.900   8883.230296   \n",
       "2015-02-28 22:00:00  8110.055916       18.900   8320.260550   \n",
       "2015-02-28 23:00:00  8519.368752       18.900   8110.055916   \n",
       "\n",
       "                     temperature_lag_1  demand_lag_2  temperature_lag_2  \\\n",
       "date_time                                                                 \n",
       "2010-12-03 00:00:00             18.325   8194.758870             18.650   \n",
       "2010-12-03 01:00:00             17.825   8311.641438             18.325   \n",
       "2010-12-03 02:00:00             17.850   7650.165828             17.825   \n",
       "2010-12-03 03:00:00             17.675   7927.140368             17.850   \n",
       "2010-12-03 04:00:00             17.625   7327.146056             17.675   \n",
       "...                                ...           ...                ...   \n",
       "2015-02-28 19:00:00             30.850  10258.585392             31.550   \n",
       "2015-02-28 20:00:00             28.350   9979.909902             30.850   \n",
       "2015-02-28 21:00:00             22.200   9596.777060             28.350   \n",
       "2015-02-28 22:00:00             18.900   8883.230296             22.200   \n",
       "2015-02-28 23:00:00             18.900   8320.260550             18.900   \n",
       "\n",
       "                     demand_lag_3  temperature_lag_3  demand_lag_24  \\\n",
       "date_time                                                             \n",
       "2010-12-03 00:00:00   8810.225934             19.000    7594.965872   \n",
       "2010-12-03 01:00:00   8194.758870             18.650    7914.538048   \n",
       "2010-12-03 02:00:00   8311.641438             18.325    7321.428112   \n",
       "2010-12-03 03:00:00   7650.165828             17.825    7045.315052   \n",
       "2010-12-03 04:00:00   7927.140368             17.850    7396.896962   \n",
       "...                           ...                ...            ...   \n",
       "2015-02-28 19:00:00  10019.921572             31.250    9980.108798   \n",
       "2015-02-28 20:00:00  10258.585392             31.550    9411.874558   \n",
       "2015-02-28 21:00:00   9979.909902             30.850    8653.510960   \n",
       "2015-02-28 22:00:00   9596.777060             28.350    8256.683092   \n",
       "2015-02-28 23:00:00   8883.230296             22.200    8716.498334   \n",
       "\n",
       "                     temperature_lag_24  ...  demand_window_8064_kurt  \\\n",
       "date_time                                ...                            \n",
       "2010-12-03 00:00:00              18.500  ...                -0.103898   \n",
       "2010-12-03 01:00:00              18.375  ...                -0.104160   \n",
       "2010-12-03 02:00:00              17.875  ...                -0.104297   \n",
       "2010-12-03 03:00:00              17.425  ...                -0.104314   \n",
       "2010-12-03 04:00:00              17.275  ...                -0.104304   \n",
       "...                                 ...  ...                      ...   \n",
       "2015-02-28 19:00:00              19.700  ...                -0.398990   \n",
       "2015-02-28 20:00:00              18.750  ...                -0.398995   \n",
       "2015-02-28 21:00:00              18.300  ...                -0.398790   \n",
       "2015-02-28 22:00:00              18.150  ...                -0.398505   \n",
       "2015-02-28 23:00:00              17.800  ...                -0.398214   \n",
       "\n",
       "                     demand_window_8064_skew  temperature_window_8064_mean  \\\n",
       "date_time                                                                    \n",
       "2010-12-03 00:00:00                 0.422491                     15.744401   \n",
       "2010-12-03 01:00:00                 0.422422                     15.743942   \n",
       "2010-12-03 02:00:00                 0.422397                     15.743378   \n",
       "2010-12-03 03:00:00                 0.422377                     15.742823   \n",
       "2010-12-03 04:00:00                 0.422436                     15.742305   \n",
       "...                                      ...                           ...   \n",
       "2015-02-28 19:00:00                 0.291512                     15.987450   \n",
       "2015-02-28 20:00:00                 0.291267                     15.988492   \n",
       "2015-02-28 21:00:00                 0.291143                     15.988814   \n",
       "2015-02-28 22:00:00                 0.291119                     15.988796   \n",
       "2015-02-28 23:00:00                 0.291138                     15.988808   \n",
       "\n",
       "                     temperature_window_8064_std  \\\n",
       "date_time                                          \n",
       "2010-12-03 00:00:00                     5.742699   \n",
       "2010-12-03 01:00:00                     5.742385   \n",
       "2010-12-03 02:00:00                     5.741955   \n",
       "2010-12-03 03:00:00                     5.741552   \n",
       "2010-12-03 04:00:00                     5.741194   \n",
       "...                                          ...   \n",
       "2015-02-28 19:00:00                     5.252119   \n",
       "2015-02-28 20:00:00                     5.253737   \n",
       "2015-02-28 21:00:00                     5.254039   \n",
       "2015-02-28 22:00:00                     5.254028   \n",
       "2015-02-28 23:00:00                     5.254035   \n",
       "\n",
       "                     temperature_window_8064_kurt  \\\n",
       "date_time                                           \n",
       "2010-12-03 00:00:00                      0.659439   \n",
       "2010-12-03 01:00:00                      0.660363   \n",
       "2010-12-03 02:00:00                      0.661546   \n",
       "2010-12-03 03:00:00                      0.662685   \n",
       "2010-12-03 04:00:00                      0.663728   \n",
       "...                                           ...   \n",
       "2015-02-28 19:00:00                      1.053306   \n",
       "2015-02-28 20:00:00                      1.051447   \n",
       "2015-02-28 21:00:00                      1.050537   \n",
       "2015-02-28 22:00:00                      1.050578   \n",
       "2015-02-28 23:00:00                      1.050551   \n",
       "\n",
       "                     temperature_window_8064_skew  demand_expanding_mean  \\\n",
       "date_time                                                                  \n",
       "2010-12-03 00:00:00                      0.778176            9842.090580   \n",
       "2010-12-03 01:00:00                      0.778423            9841.818798   \n",
       "2010-12-03 02:00:00                      0.778705            9841.581422   \n",
       "2010-12-03 03:00:00                      0.778992            9841.269728   \n",
       "2010-12-03 04:00:00                      0.779267            9840.928560   \n",
       "...                                           ...                    ...   \n",
       "2015-02-28 19:00:00                      0.793359            9463.653128   \n",
       "2015-02-28 20:00:00                      0.793594            9463.656071   \n",
       "2015-02-28 21:00:00                      0.793438            9463.643240   \n",
       "2015-02-28 22:00:00                      0.793450            9463.617965   \n",
       "2015-02-28 23:00:00                      0.793442            9463.588045   \n",
       "\n",
       "                     demand_expanding_std  demand_expanding_kurt  \\\n",
       "date_time                                                          \n",
       "2010-12-03 00:00:00           1804.188369              -0.103898   \n",
       "2010-12-03 01:00:00           1804.241597              -0.104074   \n",
       "2010-12-03 02:00:00           1804.255693              -0.104144   \n",
       "2010-12-03 03:00:00           1804.361038              -0.104420   \n",
       "2010-12-03 04:00:00           1804.509421              -0.104740   \n",
       "...                                   ...                    ...   \n",
       "2015-02-28 19:00:00           1752.041445               0.252346   \n",
       "2015-02-28 20:00:00           1752.022191               0.252414   \n",
       "2015-02-28 21:00:00           1752.004951               0.252485   \n",
       "2015-02-28 22:00:00           1751.993833               0.252528   \n",
       "2015-02-28 23:00:00           1751.986027               0.252556   \n",
       "\n",
       "                     demand_expanding_skew  \n",
       "date_time                                   \n",
       "2010-12-03 00:00:00               0.422491  \n",
       "2010-12-03 01:00:00               0.422631  \n",
       "2010-12-03 02:00:00               0.422815  \n",
       "2010-12-03 03:00:00               0.422872  \n",
       "2010-12-03 04:00:00               0.422842  \n",
       "...                                    ...  \n",
       "2015-02-28 19:00:00               0.499886  \n",
       "2015-02-28 20:00:00               0.499886  \n",
       "2015-02-28 21:00:00               0.499911  \n",
       "2015-02-28 22:00:00               0.499947  \n",
       "2015-02-28 23:00:00               0.499983  \n",
       "\n",
       "[37176 rows x 48 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe = Pipeline(\n",
    "    [\n",
    "        (\"lag\", lag_transformer),\n",
    "        (\"rolling\", window_transformer),\n",
    "        (\"expanding\", expanding_window_transformer),\n",
    "        (\"drop_missing\", imputer)\n",
    "    ]\n",
    ")\n",
    "\n",
    "df = pipe.fit_transform(df)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d2ad1b14-92af-4d96-901c-34ffc6e37ee6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's split the data into a training set and test set\n",
    "# We'll hold the most recent day as a test set\n",
    "split_date = df.index[-1] - pd.Timedelta(\"1D\")\n",
    "df_train = df[df.index <= split_date]\n",
    "df_test =  df[df.index > split_date]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b3261666-6ed9-45f3-af81-25b9447c08f0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>demand</th>\n",
       "      <th>temperature</th>\n",
       "      <th>demand_lag_1</th>\n",
       "      <th>temperature_lag_1</th>\n",
       "      <th>demand_lag_2</th>\n",
       "      <th>temperature_lag_2</th>\n",
       "      <th>demand_lag_3</th>\n",
       "      <th>temperature_lag_3</th>\n",
       "      <th>demand_lag_24</th>\n",
       "      <th>temperature_lag_24</th>\n",
       "      <th>...</th>\n",
       "      <th>demand_window_8064_kurt</th>\n",
       "      <th>demand_window_8064_skew</th>\n",
       "      <th>temperature_window_8064_mean</th>\n",
       "      <th>temperature_window_8064_std</th>\n",
       "      <th>temperature_window_8064_kurt</th>\n",
       "      <th>temperature_window_8064_skew</th>\n",
       "      <th>demand_expanding_mean</th>\n",
       "      <th>demand_expanding_std</th>\n",
       "      <th>demand_expanding_kurt</th>\n",
       "      <th>demand_expanding_skew</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-02-27 19:00:00</th>\n",
       "      <td>9980.108798</td>\n",
       "      <td>19.70</td>\n",
       "      <td>10068.040568</td>\n",
       "      <td>20.85</td>\n",
       "      <td>10483.536412</td>\n",
       "      <td>21.55</td>\n",
       "      <td>10960.255988</td>\n",
       "      <td>22.50</td>\n",
       "      <td>10190.802512</td>\n",
       "      <td>19.70</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.401271</td>\n",
       "      <td>0.293753</td>\n",
       "      <td>15.975459</td>\n",
       "      <td>5.238756</td>\n",
       "      <td>1.063314</td>\n",
       "      <td>0.792157</td>\n",
       "      <td>9464.004063</td>\n",
       "      <td>1752.302275</td>\n",
       "      <td>0.251445</td>\n",
       "      <td>0.499568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-27 20:00:00</th>\n",
       "      <td>9411.874558</td>\n",
       "      <td>18.75</td>\n",
       "      <td>9980.108798</td>\n",
       "      <td>19.70</td>\n",
       "      <td>10068.040568</td>\n",
       "      <td>20.85</td>\n",
       "      <td>10483.536412</td>\n",
       "      <td>21.55</td>\n",
       "      <td>9610.025236</td>\n",
       "      <td>18.60</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.401475</td>\n",
       "      <td>0.293600</td>\n",
       "      <td>15.975663</td>\n",
       "      <td>5.238869</td>\n",
       "      <td>1.062868</td>\n",
       "      <td>0.792025</td>\n",
       "      <td>9464.015478</td>\n",
       "      <td>1752.284576</td>\n",
       "      <td>0.251491</td>\n",
       "      <td>0.499553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-27 21:00:00</th>\n",
       "      <td>8653.510960</td>\n",
       "      <td>18.30</td>\n",
       "      <td>9411.874558</td>\n",
       "      <td>18.75</td>\n",
       "      <td>9980.108798</td>\n",
       "      <td>19.70</td>\n",
       "      <td>10068.040568</td>\n",
       "      <td>20.85</td>\n",
       "      <td>8719.930158</td>\n",
       "      <td>17.80</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.401489</td>\n",
       "      <td>0.293441</td>\n",
       "      <td>15.975769</td>\n",
       "      <td>5.238916</td>\n",
       "      <td>1.062665</td>\n",
       "      <td>0.791956</td>\n",
       "      <td>9464.014325</td>\n",
       "      <td>1752.265215</td>\n",
       "      <td>0.251564</td>\n",
       "      <td>0.499561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-27 22:00:00</th>\n",
       "      <td>8256.683092</td>\n",
       "      <td>18.15</td>\n",
       "      <td>8653.510960</td>\n",
       "      <td>18.30</td>\n",
       "      <td>9411.874558</td>\n",
       "      <td>18.75</td>\n",
       "      <td>9980.108798</td>\n",
       "      <td>19.70</td>\n",
       "      <td>8271.486968</td>\n",
       "      <td>17.80</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.401304</td>\n",
       "      <td>0.293361</td>\n",
       "      <td>15.975905</td>\n",
       "      <td>5.238962</td>\n",
       "      <td>1.062443</td>\n",
       "      <td>0.791866</td>\n",
       "      <td>9463.996399</td>\n",
       "      <td>1752.249983</td>\n",
       "      <td>0.251627</td>\n",
       "      <td>0.499591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-27 23:00:00</th>\n",
       "      <td>8716.498334</td>\n",
       "      <td>17.80</td>\n",
       "      <td>8256.683092</td>\n",
       "      <td>18.15</td>\n",
       "      <td>8653.510960</td>\n",
       "      <td>18.30</td>\n",
       "      <td>9411.874558</td>\n",
       "      <td>18.75</td>\n",
       "      <td>8800.900636</td>\n",
       "      <td>17.65</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.401094</td>\n",
       "      <td>0.293336</td>\n",
       "      <td>15.976079</td>\n",
       "      <td>5.239011</td>\n",
       "      <td>1.062190</td>\n",
       "      <td>0.791753</td>\n",
       "      <td>9463.969697</td>\n",
       "      <td>1752.239804</td>\n",
       "      <td>0.251666</td>\n",
       "      <td>0.499627</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                          demand  temperature  demand_lag_1  \\\n",
       "date_time                                                     \n",
       "2015-02-27 19:00:00  9980.108798        19.70  10068.040568   \n",
       "2015-02-27 20:00:00  9411.874558        18.75   9980.108798   \n",
       "2015-02-27 21:00:00  8653.510960        18.30   9411.874558   \n",
       "2015-02-27 22:00:00  8256.683092        18.15   8653.510960   \n",
       "2015-02-27 23:00:00  8716.498334        17.80   8256.683092   \n",
       "\n",
       "                     temperature_lag_1  demand_lag_2  temperature_lag_2  \\\n",
       "date_time                                                                 \n",
       "2015-02-27 19:00:00              20.85  10483.536412              21.55   \n",
       "2015-02-27 20:00:00              19.70  10068.040568              20.85   \n",
       "2015-02-27 21:00:00              18.75   9980.108798              19.70   \n",
       "2015-02-27 22:00:00              18.30   9411.874558              18.75   \n",
       "2015-02-27 23:00:00              18.15   8653.510960              18.30   \n",
       "\n",
       "                     demand_lag_3  temperature_lag_3  demand_lag_24  \\\n",
       "date_time                                                             \n",
       "2015-02-27 19:00:00  10960.255988              22.50   10190.802512   \n",
       "2015-02-27 20:00:00  10483.536412              21.55    9610.025236   \n",
       "2015-02-27 21:00:00  10068.040568              20.85    8719.930158   \n",
       "2015-02-27 22:00:00   9980.108798              19.70    8271.486968   \n",
       "2015-02-27 23:00:00   9411.874558              18.75    8800.900636   \n",
       "\n",
       "                     temperature_lag_24  ...  demand_window_8064_kurt  \\\n",
       "date_time                                ...                            \n",
       "2015-02-27 19:00:00               19.70  ...                -0.401271   \n",
       "2015-02-27 20:00:00               18.60  ...                -0.401475   \n",
       "2015-02-27 21:00:00               17.80  ...                -0.401489   \n",
       "2015-02-27 22:00:00               17.80  ...                -0.401304   \n",
       "2015-02-27 23:00:00               17.65  ...                -0.401094   \n",
       "\n",
       "                     demand_window_8064_skew  temperature_window_8064_mean  \\\n",
       "date_time                                                                    \n",
       "2015-02-27 19:00:00                 0.293753                     15.975459   \n",
       "2015-02-27 20:00:00                 0.293600                     15.975663   \n",
       "2015-02-27 21:00:00                 0.293441                     15.975769   \n",
       "2015-02-27 22:00:00                 0.293361                     15.975905   \n",
       "2015-02-27 23:00:00                 0.293336                     15.976079   \n",
       "\n",
       "                     temperature_window_8064_std  \\\n",
       "date_time                                          \n",
       "2015-02-27 19:00:00                     5.238756   \n",
       "2015-02-27 20:00:00                     5.238869   \n",
       "2015-02-27 21:00:00                     5.238916   \n",
       "2015-02-27 22:00:00                     5.238962   \n",
       "2015-02-27 23:00:00                     5.239011   \n",
       "\n",
       "                     temperature_window_8064_kurt  \\\n",
       "date_time                                           \n",
       "2015-02-27 19:00:00                      1.063314   \n",
       "2015-02-27 20:00:00                      1.062868   \n",
       "2015-02-27 21:00:00                      1.062665   \n",
       "2015-02-27 22:00:00                      1.062443   \n",
       "2015-02-27 23:00:00                      1.062190   \n",
       "\n",
       "                     temperature_window_8064_skew  demand_expanding_mean  \\\n",
       "date_time                                                                  \n",
       "2015-02-27 19:00:00                      0.792157            9464.004063   \n",
       "2015-02-27 20:00:00                      0.792025            9464.015478   \n",
       "2015-02-27 21:00:00                      0.791956            9464.014325   \n",
       "2015-02-27 22:00:00                      0.791866            9463.996399   \n",
       "2015-02-27 23:00:00                      0.791753            9463.969697   \n",
       "\n",
       "                     demand_expanding_std  demand_expanding_kurt  \\\n",
       "date_time                                                          \n",
       "2015-02-27 19:00:00           1752.302275               0.251445   \n",
       "2015-02-27 20:00:00           1752.284576               0.251491   \n",
       "2015-02-27 21:00:00           1752.265215               0.251564   \n",
       "2015-02-27 22:00:00           1752.249983               0.251627   \n",
       "2015-02-27 23:00:00           1752.239804               0.251666   \n",
       "\n",
       "                     demand_expanding_skew  \n",
       "date_time                                   \n",
       "2015-02-27 19:00:00               0.499568  \n",
       "2015-02-27 20:00:00               0.499553  \n",
       "2015-02-27 21:00:00               0.499561  \n",
       "2015-02-27 22:00:00               0.499591  \n",
       "2015-02-27 23:00:00               0.499627  \n",
       "\n",
       "[5 rows x 48 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c3019f9e-b717-40d6-a6fd-8c83bb3b657d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>demand</th>\n",
       "      <th>temperature</th>\n",
       "      <th>demand_lag_1</th>\n",
       "      <th>temperature_lag_1</th>\n",
       "      <th>demand_lag_2</th>\n",
       "      <th>temperature_lag_2</th>\n",
       "      <th>demand_lag_3</th>\n",
       "      <th>temperature_lag_3</th>\n",
       "      <th>demand_lag_24</th>\n",
       "      <th>temperature_lag_24</th>\n",
       "      <th>...</th>\n",
       "      <th>demand_window_8064_kurt</th>\n",
       "      <th>demand_window_8064_skew</th>\n",
       "      <th>temperature_window_8064_mean</th>\n",
       "      <th>temperature_window_8064_std</th>\n",
       "      <th>temperature_window_8064_kurt</th>\n",
       "      <th>temperature_window_8064_skew</th>\n",
       "      <th>demand_expanding_mean</th>\n",
       "      <th>demand_expanding_std</th>\n",
       "      <th>demand_expanding_kurt</th>\n",
       "      <th>demand_expanding_skew</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-02-28 00:00:00</th>\n",
       "      <td>8003.228986</td>\n",
       "      <td>17.65</td>\n",
       "      <td>8716.498334</td>\n",
       "      <td>17.80</td>\n",
       "      <td>8256.683092</td>\n",
       "      <td>18.15</td>\n",
       "      <td>8653.510960</td>\n",
       "      <td>18.30</td>\n",
       "      <td>8121.868698</td>\n",
       "      <td>17.05</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.400972</td>\n",
       "      <td>0.293260</td>\n",
       "      <td>15.976265</td>\n",
       "      <td>5.239049</td>\n",
       "      <td>1.061962</td>\n",
       "      <td>0.791634</td>\n",
       "      <td>9463.953166</td>\n",
       "      <td>1752.223953</td>\n",
       "      <td>0.251731</td>\n",
       "      <td>0.499656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 01:00:00</th>\n",
       "      <td>7522.862620</td>\n",
       "      <td>17.20</td>\n",
       "      <td>8003.228986</td>\n",
       "      <td>17.65</td>\n",
       "      <td>8716.498334</td>\n",
       "      <td>17.80</td>\n",
       "      <td>8256.683092</td>\n",
       "      <td>18.15</td>\n",
       "      <td>7629.796248</td>\n",
       "      <td>16.55</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.400779</td>\n",
       "      <td>0.293268</td>\n",
       "      <td>15.976438</td>\n",
       "      <td>5.239082</td>\n",
       "      <td>1.061758</td>\n",
       "      <td>0.791524</td>\n",
       "      <td>9463.920861</td>\n",
       "      <td>1752.218042</td>\n",
       "      <td>0.251751</td>\n",
       "      <td>0.499693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 02:00:00</th>\n",
       "      <td>7156.310422</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7522.862620</td>\n",
       "      <td>17.20</td>\n",
       "      <td>8003.228986</td>\n",
       "      <td>17.65</td>\n",
       "      <td>8716.498334</td>\n",
       "      <td>17.80</td>\n",
       "      <td>7316.774812</td>\n",
       "      <td>15.95</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.400588</td>\n",
       "      <td>0.293403</td>\n",
       "      <td>15.976600</td>\n",
       "      <td>5.239099</td>\n",
       "      <td>1.061606</td>\n",
       "      <td>0.791426</td>\n",
       "      <td>9463.877934</td>\n",
       "      <td>1752.222443</td>\n",
       "      <td>0.251728</td>\n",
       "      <td>0.499721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 03:00:00</th>\n",
       "      <td>7074.676782</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7156.310422</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7522.862620</td>\n",
       "      <td>17.20</td>\n",
       "      <td>8003.228986</td>\n",
       "      <td>17.65</td>\n",
       "      <td>7280.901386</td>\n",
       "      <td>15.30</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.400539</td>\n",
       "      <td>0.293662</td>\n",
       "      <td>15.976804</td>\n",
       "      <td>5.239115</td>\n",
       "      <td>1.061434</td>\n",
       "      <td>0.791303</td>\n",
       "      <td>9463.826904</td>\n",
       "      <td>1752.236670</td>\n",
       "      <td>0.251676</td>\n",
       "      <td>0.499735</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-02-28 04:00:00</th>\n",
       "      <td>7204.031944</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7074.676782</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7156.310422</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7522.862620</td>\n",
       "      <td>17.20</td>\n",
       "      <td>7603.832712</td>\n",
       "      <td>14.25</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.400547</td>\n",
       "      <td>0.293967</td>\n",
       "      <td>15.977034</td>\n",
       "      <td>5.239128</td>\n",
       "      <td>1.061256</td>\n",
       "      <td>0.791167</td>\n",
       "      <td>9463.774070</td>\n",
       "      <td>1752.253314</td>\n",
       "      <td>0.251617</td>\n",
       "      <td>0.499744</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                          demand  temperature  demand_lag_1  \\\n",
       "date_time                                                     \n",
       "2015-02-28 00:00:00  8003.228986        17.65   8716.498334   \n",
       "2015-02-28 01:00:00  7522.862620        17.20   8003.228986   \n",
       "2015-02-28 02:00:00  7156.310422        17.20   7522.862620   \n",
       "2015-02-28 03:00:00  7074.676782        17.20   7156.310422   \n",
       "2015-02-28 04:00:00  7204.031944        17.20   7074.676782   \n",
       "\n",
       "                     temperature_lag_1  demand_lag_2  temperature_lag_2  \\\n",
       "date_time                                                                 \n",
       "2015-02-28 00:00:00              17.80   8256.683092              18.15   \n",
       "2015-02-28 01:00:00              17.65   8716.498334              17.80   \n",
       "2015-02-28 02:00:00              17.20   8003.228986              17.65   \n",
       "2015-02-28 03:00:00              17.20   7522.862620              17.20   \n",
       "2015-02-28 04:00:00              17.20   7156.310422              17.20   \n",
       "\n",
       "                     demand_lag_3  temperature_lag_3  demand_lag_24  \\\n",
       "date_time                                                             \n",
       "2015-02-28 00:00:00   8653.510960              18.30    8121.868698   \n",
       "2015-02-28 01:00:00   8256.683092              18.15    7629.796248   \n",
       "2015-02-28 02:00:00   8716.498334              17.80    7316.774812   \n",
       "2015-02-28 03:00:00   8003.228986              17.65    7280.901386   \n",
       "2015-02-28 04:00:00   7522.862620              17.20    7603.832712   \n",
       "\n",
       "                     temperature_lag_24  ...  demand_window_8064_kurt  \\\n",
       "date_time                                ...                            \n",
       "2015-02-28 00:00:00               17.05  ...                -0.400972   \n",
       "2015-02-28 01:00:00               16.55  ...                -0.400779   \n",
       "2015-02-28 02:00:00               15.95  ...                -0.400588   \n",
       "2015-02-28 03:00:00               15.30  ...                -0.400539   \n",
       "2015-02-28 04:00:00               14.25  ...                -0.400547   \n",
       "\n",
       "                     demand_window_8064_skew  temperature_window_8064_mean  \\\n",
       "date_time                                                                    \n",
       "2015-02-28 00:00:00                 0.293260                     15.976265   \n",
       "2015-02-28 01:00:00                 0.293268                     15.976438   \n",
       "2015-02-28 02:00:00                 0.293403                     15.976600   \n",
       "2015-02-28 03:00:00                 0.293662                     15.976804   \n",
       "2015-02-28 04:00:00                 0.293967                     15.977034   \n",
       "\n",
       "                     temperature_window_8064_std  \\\n",
       "date_time                                          \n",
       "2015-02-28 00:00:00                     5.239049   \n",
       "2015-02-28 01:00:00                     5.239082   \n",
       "2015-02-28 02:00:00                     5.239099   \n",
       "2015-02-28 03:00:00                     5.239115   \n",
       "2015-02-28 04:00:00                     5.239128   \n",
       "\n",
       "                     temperature_window_8064_kurt  \\\n",
       "date_time                                           \n",
       "2015-02-28 00:00:00                      1.061962   \n",
       "2015-02-28 01:00:00                      1.061758   \n",
       "2015-02-28 02:00:00                      1.061606   \n",
       "2015-02-28 03:00:00                      1.061434   \n",
       "2015-02-28 04:00:00                      1.061256   \n",
       "\n",
       "                     temperature_window_8064_skew  demand_expanding_mean  \\\n",
       "date_time                                                                  \n",
       "2015-02-28 00:00:00                      0.791634            9463.953166   \n",
       "2015-02-28 01:00:00                      0.791524            9463.920861   \n",
       "2015-02-28 02:00:00                      0.791426            9463.877934   \n",
       "2015-02-28 03:00:00                      0.791303            9463.826904   \n",
       "2015-02-28 04:00:00                      0.791167            9463.774070   \n",
       "\n",
       "                     demand_expanding_std  demand_expanding_kurt  \\\n",
       "date_time                                                          \n",
       "2015-02-28 00:00:00           1752.223953               0.251731   \n",
       "2015-02-28 01:00:00           1752.218042               0.251751   \n",
       "2015-02-28 02:00:00           1752.222443               0.251728   \n",
       "2015-02-28 03:00:00           1752.236670               0.251676   \n",
       "2015-02-28 04:00:00           1752.253314               0.251617   \n",
       "\n",
       "                     demand_expanding_skew  \n",
       "date_time                                   \n",
       "2015-02-28 00:00:00               0.499656  \n",
       "2015-02-28 01:00:00               0.499693  \n",
       "2015-02-28 02:00:00               0.499721  \n",
       "2015-02-28 03:00:00               0.499735  \n",
       "2015-02-28 04:00:00               0.499744  \n",
       "\n",
       "[5 rows x 48 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fe718e6-af6e-4b8a-9943-a806cd6850fd",
   "metadata": {},
   "source": [
    "# Use LASSO to select features"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d3c3402-23ce-46a2-9584-b05e621e31a4",
   "metadata": {},
   "source": [
    "Let's create the target and features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "753d7965-34db-457a-b763-cd773f43616c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create target variable\n",
    "y_train = df_train[\"demand\"]\n",
    "\n",
    "# Drop demand and temperature as features, we do not know them at predict time.\n",
    "X_train = df_train.drop(columns=[\"demand\", \"temperature\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0664a776-0c86-42bb-b70e-e396cf3e4da6",
   "metadata": {},
   "source": [
    "We will apply standard scaling because we are using LASSO.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "22e1014a-fb8e-48b9-bfd0-4274bf44e19b",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_ = StandardScaler().fit_transform(X_train)\n",
    "X_train = pd.DataFrame(data=X_train_, columns=X_train.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7f916b96-06e7-4ce4-a0b3-6fdd0e845db9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "dbaebed4-302f-44a7-b57e-0ed7533fde96",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Lasso(alpha=1, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Lasso</label><div class=\"sk-toggleable__content\"><pre>Lasso(alpha=1, random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "Lasso(alpha=1, random_state=0)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = Lasso(alpha=1, random_state=0)\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0013c6f3-77ae-418a-bc08-ea002b425a8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_importances = pd.Series(index=X_train.columns, data=model.coef_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "4b72b9c6-85c0-4de2-ab9b-ec764761eebc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "demand_lag_1                    2190.943047\n",
       "demand_lag_2                     877.510005\n",
       "demand_lag_168                   213.888675\n",
       "temperature_lag_1                190.948998\n",
       "demand_window_24_mean            147.480480\n",
       "temperature_lag_3                103.812987\n",
       "demand_lag_24                    102.349227\n",
       "demand_window_24_skew             63.210775\n",
       "temperature_lag_168               55.132948\n",
       "demand_window_168_mean            41.089953\n",
       "demand_lag_3                      36.014360\n",
       "temperature_window_24_mean        32.886558\n",
       "demand_window_672_mean            21.355831\n",
       "temperature_window_24_std         17.670526\n",
       "temperature_window_24_skew        13.614140\n",
       "temperature_lag_24                12.771168\n",
       "demand_expanding_kurt              9.456486\n",
       "demand_expanding_mean              7.193448\n",
       "demand_window_672_kurt             5.706723\n",
       "temperature_window_24_kurt         5.626029\n",
       "temperature_window_168_skew        4.992910\n",
       "temperature_lag_2                  4.831044\n",
       "temperature_window_8064_std        4.226003\n",
       "temperature_window_8064_mean       4.224539\n",
       "temperature_window_672_mean        3.873723\n",
       "demand_expanding_std               3.827634\n",
       "demand_expanding_skew              3.613087\n",
       "demand_window_168_skew             3.234699\n",
       "demand_window_672_std              2.144170\n",
       "temperature_window_672_skew        1.069981\n",
       "demand_window_24_kurt              0.910350\n",
       "demand_window_8064_skew            0.000000\n",
       "demand_window_24_std               0.000000\n",
       "demand_window_168_std              0.000000\n",
       "temperature_window_8064_skew       0.000000\n",
       "temperature_window_8064_kurt       0.000000\n",
       "demand_window_168_kurt             0.000000\n",
       "demand_window_8064_kurt            0.000000\n",
       "temperature_window_168_kurt        0.000000\n",
       "demand_window_8064_std             0.000000\n",
       "demand_window_8064_mean            0.000000\n",
       "temperature_window_672_kurt        0.000000\n",
       "temperature_window_672_std         0.000000\n",
       "temperature_window_168_mean        0.000000\n",
       "demand_window_672_skew             0.000000\n",
       "temperature_window_168_std         0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_importances.abs().sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c72bc42-a897-4f58-95d2-3c6ff7dbb997",
   "metadata": {},
   "source": [
    "We can see that the lag features are most important but some window features are also selected! This is to be expected as the most recent observations tend to be very predictive of the next immediate observation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
